Method and apparatus for user interaction data storage

ABSTRACT

A method on an electronic device is described. A touch screen display of the electronic device is actively controlled in a higher-power mode of operation. Actively controlling the touch screen display in the higher-power mode is discontinued to enter a lower-power mode of operation. In the lower-power mode: at least one first control signal is provided to the touch screen display; in response to the at least one first control signal, a first portion of the touch screen display is activated and a first portion of a graphic is displayed on a first area of the touch screen display within the first portion; occurrence of a first user interaction that corresponds to the first portion of the graphic during the display of the first portion of the graphic is determined; and user interaction data is stored for the first portion of the graphic based on the first user interaction determination.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.13/964,034, filed Aug. 9, 2013, the entire content of which is herebyincorporated by reference.

TECHNICAL FIELD

The disclosure relates to user interaction with an electronic device.

BACKGROUND

The traditional “Notification LED” on an electronic device has lost itsusefulness for users who often receive a constant stream ofnotifications. For such users, the notification LED blinks often andthus becomes “noise” that provides little in the way of usefulinformation and, indeed, an always-blinking LED drowns out importantnotifications. Additionally, devices employing such notification LEDsfoster an operational paradigm of “fidgeting” in which the user mustconstantly wake the device simply to view each notification. Suchoperation is both time consuming and cognitively draining. In view ofsuch concerns, and others, it would be advantageous if one or moreimproved methods of providing notifications to users of electronicdevices, and improved electronic devices configured to perform suchmethods, could be developed.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other embodiments will become apparent and more readilyappreciated from the following description of the embodiments, taken inconjunction with the accompanying drawings of which:

FIG. 1 is a perspective view of an example embodiment of an electronicdevice in accordance with the present disclosure;

FIG. 2 is a block diagram representing example components of theelectronic device of FIG. 1;

FIGS. 3A, 3B, 3C, and 3D are four example views of a display screen ofthe electronic device of FIG. 1 during four different points ofoperation of a method for display;

FIGS. 4A, 4B, 4C, and 4D respectively show the example views of FIGS.3A, 3B, 3C, and 3D of the display screen, respectively, in combinationwith illustrations of a user interacting with the display screen;

FIG. 5 is a flowchart illustrating one embodiment of a method forstoring user interaction data and performing an action by the electronicdevice of FIG. 1;

FIGS. 6A, 6B, 6C, and 6D are four example views of a display screen ofthe electronic device of FIG. 1 during operation of the method of FIG. 5illustrating a survey for user interaction;

FIGS. 7A, 7B, and 7C are three example views of a display screen of theelectronic device of FIG. 1 during operation of the method of FIG. 5illustrating available actions;

FIG. 8 is a diagram of one example of user interaction data stored byone or more of the electronic devices of FIG. 1;

FIG. 9 is a chart representing an example histogram for view time of agraphic.

DETAILED DESCRIPTION

A variety of enhancements can be provided to electronic devices so as toenable to the electronic devices to better provide notifications tousers, and to the methods of operation of electronic devices inproviding notifications. Although mobile device displays and indeed theapplication processors of mobile devices are in a powered-down or “off”state much or most of the time (e.g., in a “sleep mode”), so as to saveon power consumed by such devices, such an “off” state of the displaypresents an opportunity for displaying the most relevant information atjust the right time. Further, because a user's attention is a limitedresource, in addition to showing the right information at the righttime, displaying of notifications in this manner during the “off” statecould also help reduce the amount of time it takes to access the mostimportant notifications. By giving users the right amount of informationat the right time, the users will be better informed to decide whetheror not to pick up and actively use (unlock or turn “on”) their device—orto simply dismiss such notifications without unlocking their device.

Accordingly, it is possible to allow for ongoing activation of a mobiledevice display, even while the mobile device application processor anddisplay are technically in a powered-down or “off” state, byimplementing Smart Partial OLED display technology and a sensor hub (orother auxiliary processing device) for controlling a display using suchtechnology, rather than using the mobile device application processor,so as to achieve the displaying of notifications in a low-power (orlower-power) manner relative to normal “on” state operation of a displayby way of the mobile device application processor. Embodiments operatingin this manner can be considered as providing “always-on” operationinsofar as, even though the mobile device (or at least the mobile deviceapplication processor) is powered-down or off, the display can still beactivated to display notifications when appropriate. That is, touchinputs can be received, while the display is in a lower power state thatis more akin to an “off” state in terms of power usage, insofar as thedisplay is “on” only periodically or otherwise to a limited extent. Suchoperation can be advantageous both in terms of reducing the rate ofbattery depletion, as well as reducing the chance that portions of adisplay might be overly illuminated for too great of time in a mannerthat could damage the display.

In at least some embodiments, the display of the “right information atthe right time” is achieved by leveraging the Smart Partial OLED displaysystem to display an optimized set of notifications while the screen is“off”, where optimization can be achieved for example by way ofintelligent prioritization/ranking based on various information such asvarious inputs/behaviors, and improved notification management. In thismanner, the most important contextually-relevant information reaches theuser, and particularly can be provided to the user when the user has aneed for that information (in such cases, there can also be integrationwith personal assistant-type applications such as Google Now availablefrom Google, Inc. of Mountain View, Calif.).

Also, in at least some embodiments, the “always-on” operation providesusers with a seamless or substantially seamless experience as the usersutilize the device and the device transitions between the “screen off”state (lock screen) and the “on” state. In at least some exampleembodiments, the device comes alive while still in the “off” mode (e.g.,where the application processor is still dormant) by showing incomingnotifications as they occur, by providing “breathing” alerts on thescreen without substantially draining the battery, and by coming alivewhen the device is picked up off of a surface such as a table. Also, insome example embodiments, the providing of notifications is entirelydisabled by the device when the device recognizes a circumstance wherethe providing of notifications is inappropriate (e.g., when the deviceis face down, when the device is in a pocket, when it is nighttime,etc.).

Further, in at least some embodiments, the low-power operatingcapability of the OLED display during the “off” state of the device isfurther leveraged by operating the display in a manner by whichimportant notifications are shown only on a small part of the screen. Insome cases, not only does the display system display an optimized set ofnotifications while the screen is “off”, without significantly impactingbattery life, but also the system allows for a select set of userinteractions during this state, which also avoid significant impact uponbattery life. Also, in at least some cases, there can further be acontinuous, cohesive experience between phone and watch operation. Also,at least some embodiments can provide a user-centric approach tosecurity.

In addition to notifications to a user, advertisements to the user onmobile devices are also common. However, these advertisements aretypically placed within applications or the operating systems and thusmust be viewed when the user is actively using the mobile device. Theadvertisement must then compete for the user's attention with theapplications or operating system. Because the user is more likely todirect their attention to the application than the advertisement, anadvertiser that pays for the advertisement placement is less likely topay a higher fee for the advertisement's placement. The use of an“always-on” operation for display of graphics or advertisements alongwith user interaction during their display allows for more targetedadvertising and information gathering.

The present disclosure is generally directed to a method and system forstoring user interaction data. In particular, the disclosure is directedto storing user interaction data for advertisements on an electronicdevice.

The present disclosure describes a method of an electronic device. Atouch screen display of the electronic device is actively controlled ina higher-power mode of operation. Actively controlling the touch screendisplay in the higher-power mode of operation is discontinued to enter alower-power mode of operation. In the lower-power mode of operation, atleast one first control signal is provided to the touch screen display.In the lower-power mode of operation and in response to the at least onefirst control signal, a first portion of the touch screen display isactivated. In the lower-power mode of operation and in response to theat least one first control signal, a first portion of a graphic isdisplayed on a first area of the touch screen display that is within thefirst portion of the touch screen display. In the lower-power mode ofoperation, whether a first user interaction that corresponds to thefirst portion of the graphic has occurred during the display of thefirst portion of the graphic is determined. In the lower-power mode ofoperation, user interaction data is stored for the first portion of thegraphic based on the first user interaction determination.

The present disclosure further describes an electronic device. Theelectronic device includes a receiver configured to receive wirelesscommunication signals, at least one memory device, a touch screendisplay, at least one application processor configured to control thetouch screen display in a higher-power mode of operation, and at leastone processing device configured to control the touch screen display ina lower-power mode of operation. The at least one application processoris configured to receive and store a graphic in the at least one memorydevice in the higher-power mode of operation. The electronic device isconfigured to discontinue actively controlling the touch screen displayin the higher-power mode of operation with the application processor toenter the lower-power mode of operation. The at least one processingdevice is configured to provide, in the lower-power mode of operation,at least one first control signal to the touch screen display. The touchscreen display is configured to activate, in the lower-power mode ofoperation and in response to the at least one first control signal, afirst portion of the touch screen display. The touch screen display isconfigured to display, in the lower-power mode of operation and inresponse to the at least one first control signal, a first portion ofthe graphic on a first area of the touch screen display that is withinthe first portion of the touch screen display. The at least oneprocessing device is configured to determine, in the lower-power mode ofoperation, whether a first touch input that corresponds to the firstarea has occurred during the display of the first portion of thegraphic. The at least one processing device is configured to provide, inthe lower-power mode of operation and in response to the first touchinput, at least one second control signal to the touch screen display.The touch screen display is configured to activate, in the lower-powermode of operation and in response to the at least one second controlsignal, a second portion of the touch screen display. The touch screendisplay is configured to display, in the lower-power mode of operationand in response to the at least one second control signal, a secondportion of the graphic on a second area of the touch screen display thatis within the second portion of the touch screen display. The at leastone processing device is configured to store, in the lower-power mode ofoperation, user interaction data for the graphic. The user interactiondata includes at least one of a duration of the first touch input, aduration of the display of the second portion of the graphic, or anaction indication that corresponds to the first touch input.

The present disclosure further describes another electronic device. Theelectronic device includes a touch screen display, an applicationprocessor configured to control the touch screen display in ahigher-power mode of operation, a sensor hub configured to control thetouch screen display in an lower-power mode of operation, and anon-transitory memory. The electronic device is configured to activelycontrol the touch screen display in the higher-power mode of operationwith the application processor. The electronic device is configured todiscontinue actively controlling the touch screen display in thehigher-power mode of operation with the application processor to enterthe lower-power mode of operation. The sensor hub is configured toprovide, in the lower-power mode of operation, at least one firstcontrol signal to the touch screen display. The touch screen display isconfigured to activate, in the lower-power mode of operation and inresponse to the at least one first control signal, a first portion ofthe touch screen display. The touch screen display is configured todisplay, in the lower-power mode of operation and in response to the atleast one first control signal, a first portion of a graphic on a firstarea of the touch screen display that is within the first portion of thetouch screen display. The sensor hub is configured to determine, in thelower-power mode of operation, whether a first user interaction thatcorresponds to the first portion of the graphic has occurred during thedisplay of the first portion of the graphic. The sensor hub isconfigured to store, in the lower-power mode of operation, userinteraction data for the first portion of the graphic based on the firstuser interaction determination.

Turning to FIG. 1, there is illustrated a perspective view of an exampleelectronic device 100. In the present embodiment, the electronic device100 can be any type of device capable of providing touch screeninteractive capabilities. Examples of the electronic device 100 include,but are not limited to, mobile devices, wireless devices, smart phones,tablet computing devices, personal digital assistants, personalnavigation devices, touch screen input devices, touch or pen-based inputdevices, portable video or audio players, as well as any of a variety ofother electronic devices. It is to be understood that the electronicdevice 100 may take the form of a variety of form factors, such as, butnot limited to, bar, tablet, flip/clam, slider and rotator form factors.

In the present example embodiment shown in FIG. 1, the electronic device100 has a housing 101 comprising a front surface 103 which includes atouch screen display (that is a visible display) 105 that, together withcertain other features discussed below, constitutes a user interface. Inthe present example, the touch screen display 105 is a touch screenincluding a touch-sensitive surface that overlays a display surfaceforming part of (or that is positioned just underneath or inwardly of)the front surface 103. In the present embodiment, the touch screendisplay 105 (and particularly the display surface thereof) employsorganic light-emitting diode (OLED) technology. Further, the userinterface of the electronic device 100 as shown can also include one ormore input keys 107. Examples of the input key or keys 107 include, butare not limited to, keys of an alpha or numeric (e.g., alphanumeric)keypad (or other keyboard), physical keys, touch-sensitive surfaces,mechanical surfaces, multipoint directional keys, and side buttons orkeys. Further as shown, the electronic device 100 can also comprise aspeaker 109 and microphone 111 for audio output and input, respectively,along the front surface 103 (or other outer surfaces of the device).

Notwithstanding the particular features shown in FIG. 1, in an alternateembodiment, the electronic device can include other features. Forexample, in place of the touch screen display, in an alternateembodiment the electronic device can employ a touch-sensitive surfacesupported by the housing 101 that does not overlay (or is not overlaidby) any type of display. Indeed, although FIG. 1 shows particularlyexample display and user interface features, it is to be understood thatthe electronic device 100 can include a variety of other combinations ofdisplay and user interface features depending upon the embodiment.

Additionally as shown in FIG. 1, the electronic device 100 includes oneor more sensors 113, a number of which are shown to be positioned at orwithin an exterior boundary of the housing 101 (and can be supported onor within the housing 101). More particularly, as illustrated by FIG. 1,in the present embodiment the sensor or sensors 113 can be positioned atthe front surface 103, another surface (such as one or more sidesurfaces 115) of the exterior boundary of the housing 101, or both. Inthe present embodiment, at least some of the sensors 113 (whether at theexterior boundary or within the exterior boundary, i.e., internal to thehousing) are configured to detect one or more predeterminedenvironmental conditions associated with an environment external orinternal to the housing. Further examples of the sensors 113 aredescribed below in reference to FIG. 2.

Referring to FIG. 2, there is shown a block diagram representing examplecomponents (e.g., internal components) 200 of the electronic device 100of FIG. 1. In the present embodiment, the components 200 include one ormore wireless transceivers 201, one or more processors 203, one or morememories 205, one or more output components 207, and one or more inputcomponents 209. As already noted above, the electronic device 100includes a user interface, including the touch screen display 105 thatcomprises one or more of the output components 207 and one or more ofthe input components 209. Also as already discussed above, theelectronic device 100 includes a plurality of the sensors 113, severalof which are described in more detail below. In the present embodiment,the sensors 113 are in communication with (so as to provide sensorsignals to or receive control signals from) a sensor hub 224.

Further, the components 200 include a device interface 215 to provide adirect connection to auxiliary components or accessories for additionalor enhanced functionality. In addition, the internal components 200include a power source or supply 217, such as a portable battery, forproviding power to the other internal components and allow portabilityof the electronic device 100. As shown, all of the components 200, andparticularly the wireless transceivers 201, processors 203, memories205, output components 207, input components 209, sensor hub 224, deviceinterface 215, and power supply 217, are coupled directly or indirectlywith one another by way of one or more internal communication link(s)218 (e.g., an internal communications bus).

Further, in the present embodiment of FIG. 2, the wireless transceivers201 particularly include a cellular transceiver 211 and a Wi-Fitransceiver 213. Although in the present embodiment the wirelesstransceivers 201 particularly include two of the wireless transceivers211 and 213, the present disclosure is intended to encompass numerousembodiments in which any arbitrary number of (e.g., more than two)wireless transceivers employing any arbitrary number of (e.g., two ormore) communication technologies are present. More particularly, in thepresent embodiment, the cellular transceiver 211 is configured toconduct cellular communications, such as 3G, 4G, 4G-LTE, vis-à-vis celltowers (not shown), albeit in other embodiments, the cellulartransceiver 211 can be configured to utilize any of a variety of othercellular-based communication technologies such as analog communications(using AMPS), digital communications (using CDMA, TDMA, GSM, iDEN, GPRS,EDGE, etc.), or next generation communications (using UMTS, WCDMA, LTE,IEEE 802.16, etc.) or variants thereof.

By contrast, the Wi-Fi transceiver 213 is a wireless local area network(WLAN) transceiver configured to conduct Wi-Fi communications inaccordance with the IEEE 802.11 (a, b, g, or n) standard with accesspoints. In other embodiments, the Wi-Fi transceiver 213 can instead (orin addition) conduct other types of communications commonly understoodas being encompassed within Wi-Fi communications such as some types ofpeer-to-peer (e.g., Wi-Fi Peer-to-Peer) communications. Further, inother embodiments, the Wi-Fi transceiver 213 can be replaced orsupplemented with one or more other wireless transceivers configured fornon-cellular wireless communications including, for example, wirelesstransceivers employing ad hoc communication technologies such as HomeRF(radio frequency), Home Node B (3G femtocell), Bluetooth, or otherwireless communication technologies such as infrared technology.Although in the present embodiment each of the wireless transceivers 201serves as or includes both a respective transmitter and a respectivereceiver, it should be appreciated that the wireless transceivers arealso intended to encompass one or more receiver(s) that are distinctfrom any transmitter(s), as well as one or more transmitter(s) that aredistinct from any receiver(s). In one example embodiment encompassedherein, the wireless transceiver 201 includes at least one receiver thatis a baseband receiver.

Exemplary operation of the wireless transceivers 201 in conjunction withothers of the components 200 of the electronic device 100 can take avariety of forms and can include, for example, operation in which, uponreception of wireless signals (as provided, for example, by remotedevice(s)), the internal components detect communication signals and thetransceivers 201 demodulate the communication signals to recoverincoming information, such as voice or data, transmitted by the wirelesssignals. After receiving the incoming information from the transceivers201, the processors 203 format the incoming information for the one ormore output components 207. Likewise, for transmission of wirelesssignals, the processors 203 format outgoing information, which can butneed not be activated by the input components 209, and convey theoutgoing information to one or more of the wireless transceivers 201 formodulation so as to provide modulated communication signals to betransmitted. The wireless transceiver(s) 201 convey the modulatedcommunication signals by way of wireless (as well as possibly wired)communication links to other devices (e.g., remote devices). Thewireless transceivers 201 in one example allow the electronic device 100to exchange messages with remote devices, for example, a remote networkentity 214 of a cellular network or WLAN network. Examples of the remotenetwork entity 214 include an application server, web server, databaseserver, or other network entity accessible through the wirelesstransceivers 201 either directly or indirectly via one or moreintermediate devices or networks (e.g., via a WLAN access point, theInternet, LTE network, or other network).

Depending upon the embodiment, the output and input components 207, 209of the components 200 can include a variety of visual, audio, ormechanical outputs. For example, the output device(s) 207 can includeone or more visual output devices such as a cathode ray tube, liquidcrystal display, plasma display, video screen, incandescent light,fluorescent light, front or rear projection display, and light emittingdiode indicator, one or more audio output devices such as a speaker,alarm, or buzzer, or one or more mechanical output devices such as avibrating mechanism or motion-based mechanism. Likewise, by example, theinput device(s) 209 can include one or more visual input devices such asan optical sensor (for example, a camera lens and photosensor), one ormore audio input devices such as a microphone, and one or moremechanical input devices such as a flip sensor, keyboard, keypad,selection button, navigation cluster, touch pad, capacitive sensor,motion sensor, and switch.

As noted, the user interface and particularly the touch screen display105 of the electronic device 100 of FIG. 1 can be considered toconstitute or include both one or more of the input components 209,particularly a touch-sensitive input component 219 shown in FIG. 2, andone or more of the output components 207. Further, it should beappreciated that the operations that can actuate one or more of theinput devices 209 can include not only the physical pressing/activatingof the touch screen display 105 or buttons or other actuators of theuser interface or otherwise, but can also include, for example, openingthe electronic device 100 (if it can take on open or closed positions),unlocking the electronic device 100, moving the electronic device toactuate a motion, moving the electronic device to actuate a locationpositioning system, and operating the electronic device.

In the present embodiment, one or more of the input components 209, suchas one or more input components encompassed by the user interface suchas the touch-sensitive component 219 shown in FIG. 1, can produce aninput signal in response to detecting a predetermined gesture. In thisregard, the touch-sensitive component 219 can be considered a gesturesensor and can be or include, for example, a touch-sensitive sensorhaving a touch-sensitive surface substantially parallel to the display.The touch-sensitive sensor can include at least one of a capacitivetouch sensor, a resistive touch sensor, an acoustic sensor, anultrasonic sensor, a proximity sensor, or an optical sensor.

As mentioned above, the components 200 also can include one or more ofvarious types of the sensors 113. Although the sensors 113 are for thepurposes of FIG. 2 shown to be distinct from the input devices 209, thesensors can also be considered as being encompassed among the inputdevices 209. In alternate embodiments, one or more of the input devicescan be encompassed among the sensors, one or more of the sensors can beconsidered distinct from the input devices, one or more of the inputdevices can be considered distinct from the sensors, or all of thesensors can be considered distinct from all of the input devices andvice-versa.

With respect to the sensors 113 particularly shown in FIG. 2, theseparticularly include various sensors 225 through 231 that are examplesof sensors that can be included or utilized by the electronic device100. As already noted, as shown in FIG. 2, the various sensors 225-231in the present embodiment can be controlled by the sensor hub 224, whichcan operate in response to or independent of the processor(s) 203. Thevarious sensors 225 through 231 can include, but are not limited to, oneor more power sensors 225, one or more temperature sensors 227, one ormore pressure sensors 228, one or more moisture sensors 229, and one ormore ambient noise sensors 231.

Further in regard to the present embodiment, and as discussed furtherbelow, it should be understood that the sensor hub 224, in addition tocontrolling the various sensors 225 through 231, also serves to controloperation of the touch screen display 105 of the user interface (and thefunctionality that supports it) when the electronic device 100 andparticularly the touch screen display is considered to be “off”,including times at which the electronic device is operating in anintermediate mode or “breathing mode” of operation as described below.This is in contrast to times at which the electronic device 100 is awakeor “on”, during which times the touch screen display 105 of the userinterface (and the functionality that supports it) is under the controlof the processor(s) 203, which can be considered applicationprocessor(s). This manner of operation involving control by the sensorhub 224 at times when the electronic device is “off” is advantageousbecause the sensor hub consumes substantially less power than theprocessors 203 therefore operation under the control of the sensor hubcan be provided with substantially less battery drain than operationunder the processor(s) 203.

Although the various sensors 225 through 231 are shown in FIG. 2, inother embodiments one or more of numerous other types of sensors canalso be included among the sensors 113 including, for example, one ormore motion sensors, including for example one or more accelerometers orGyro sensors (not shown), one or more light sensors, one or moreproximity sensors (e.g., a light detecting sensor, an ultrasoundtransceiver or an infrared transceiver), one or more other touchsensors, one or more altitude sensors, one or more locationcircuits/components that can include, for example, a Global PositioningSystem (GPS) receiver, a triangulation receiver, an accelerometer, atilt sensor, a gyroscope, or any other information collecting devicethat can identify a current location or user-device interface (carrymode) of the electronic device 100.

With respect to the processor(s) 203, the processor(s) can include anyone or more processing or control devices such as, for example, amicroprocessor, microcomputer, application-specific integrated circuit,etc. The processors 203 can generate commands, for example, based oninformation received from the one or more input components 209. Theprocessor(s) 203 can process the received information alone or incombination with other data, such as information stored in the memories205. Thus, the memories 205 of the components 200 can be used by theprocessors 203 to store and retrieve data.

Further, the memories (or memory portions) 205 of the components 200 canencompass one or more memory devices of any of a variety of forms (e.g.,read-only memory, random access memory, static random access memory,dynamic random access memory, etc.), and can be used by the processors203 to store and retrieve data. In some embodiments, one or more of thememories 205 can be integrated with one or more of the processors 203 ina single device (e.g., a processing device including memory orprocessor-in-memory (PIM)), albeit such a single device will stilltypically have distinct portions/sections that perform the differentprocessing and memory functions and that can be considered separatedevices. The data that is stored by the memories 205 can include, butneed not be limited to, operating systems, applications, andinformational data.

Each operating system includes executable code that controls basicfunctions of the electronic device 100, such as interaction among thevarious components included among the components 200, communication withexternal devices or networks via the wireless transceivers 201 or thedevice interface 215, and storage and retrieval of applications anddata, to and from the memories 205. Each application includes executablecode that utilizes an operating system to provide more specificfunctionality, such as file system service and handling of protected andunprotected data stored in the memories 205. Such operating system orapplication information can include software update information (whichcan be understood to potentially encompass updates to eitherapplication(s) or operating system(s) or both). As for informationaldata, this is non-executable code or information that can be referencedor manipulated by an operating system or application for performingfunctions of the electronic device 100.

It is to be understood that FIG. 2 is provided for illustrative purposesonly and for illustrating components of an electronic device inaccordance with various embodiments, and is not intended to be acomplete schematic diagram of the various components required for anelectronic device. Therefore, an electronic device can include variousother components not shown in FIG. 2, or can include a combination oftwo or more components or a division of a particular component into twoor more separate components, and still be within the scope of thedisclosed embodiments.

The electronic device 100 in one embodiment is configured to proceedamong different operational modes or states, mainly, a “fully awake” or“on” mode of operation during which the processors 203 are operating(that is, the application processors are up and running), an “off” modeof operation, and an “intermediate” mode of operation during which“breathing” and “peeking” processes occur or can occur. Although in thepresent embodiment the processors 203 are off or powered-down during theoff mode and intermediate mode of operation (by contrast to operation inthe fully awake mode, during which the processors 203 are up andrunning), as will be described further below it is still possible forthe electronic device 100 to operate during one or both of the off modeand intermediate mode in manners that involve some display functionalityby the touch screen display 105 (including potentially both outputtingof information and receiving of inputs). Such operation of the touchscreen display 105 during the intermediate mode or off mode is achievedby virtue of control functionality provided by the sensor hub 224.

It should further be appreciated that, generally speaking, operation inthe fully awake mode is associated with a normal power mode of theprocessors 203 (application processors) or a substantially active modeof those processors. By contrast, the intermediate mode of operation andoff mode of operation are associated with a low power (or even completedpowered-off) mode of the processors 203 (application processors) or asubstantially inactive mode of those processors. Given the above, theintermediate mode of operation can also be referred to as an “Always onDisplay” mode (AoD mode), “sleep” mode (or the off mode potentially canalso be encompassed generally within the “sleep” mode), or lower-powermode of operation, in contrast to the fully awake mode, which can bereferred to as a “non-sleep” mode or higher-power mode of operation.

Additionally as will be described further below, in at least someembodiments disclosed herein, one or more notifications can be displayedby the electronic device 100 in a “non-sleep” mode such as the fullyawake mode and also one or more notifications can be displayed by theelectronic device in a “sleep” mode such as the intermediate or AoD mode(including submode portions thereof as described). In at least some suchembodiments, the notifications provided in the fully awake mode or“non-sleep” mode are notifications received during operation in thatmode, and that are displayed by way of activation of the entire, orsubstantially the entire, display screen (e.g., full screen informationis displayed, where the full screen information includes datacorresponding to substantially all pixels capable of being displayed bythe display). Further in such embodiments, in contrast, thenotifications provided in the intermediate mode or “sleep” mode arenotifications received during operation in that mode, and that aredisplayed by way of activation of only a portion or portions of thedisplay screen (particularly portion(s) that are substantially less thanthe entire display screen or display substantially less that the fullscreen information, that is, information corresponding to less orsubstantially less than all pixels capable of being displayed by thedisplay).

Turning to FIGS. 3A, 3B, 3C, and 3D, respectively, first, second, third,and fourth example views of the touch screen display 105 of theelectronic device 100 are shown that are intended to illustrate exampleoperation of the touch screen display in displaying images or graphicsfor a user interaction. FIG. 3A particularly shows a blank image 300that appears when the touch screen display 105 is completely off.

By contrast, FIG. 3B shows a breathing view image 302 that, rather thanbeing blank, instead includes one or more image portions that aredisplayed by the touch screen display 105, and which in the presentexample particularly include an icon 304 that can constitute a hit areathat can be touched by a user during operation in the intermediate mode,additional icons 306 that also can constitute an additional hit area (orareas), and a time display 308. The breathing view image 302 does notremain consistently on at all times during operation in the intermediatemode, but rather periodically becomes visible and then disappears (atwhich times the touch screen display 105 again takes on the blank image300 of FIG. 3A.

Next, with respect to FIG. 3C, a peek animation image 310 isillustrated. As shown, the peek animation image 310 also includes one ormore image portions that are displayed by the touch screen display 105,and more particularly in this embodiment these one or more imageportions include not only all of the image portions shown in FIG. 3B(namely, icon 304, icons 306 and time display 308) but also additionallyinclude animation features 312, which in the present example includeupper and lower (that is, above the time display 308 and below the icons306) vertically-extending columns of three dots. As further discussedbelow, the animation features 312 particularly are displayed by thetouch screen display 105 during transitioning between a time during theintermediate mode at which a touch can be received (e.g., a time atwhich the breathing view image 302 shown in FIG. 3B is displayed), and atime at which the peek view mode has been fully entered and a peek viewimage such as an example image shown in FIG. 3D is displayed.

Additionally, with respect FIG. 3D, the peek view image 314 showntherein is an example of an image that can be displayed by the touchscreen display 105 once peek view mode has been entered. As shown, thepeek view image 314 no longer includes the exact same arrangement ofimage portions shown in the peek animation view 310, albeit some of theimage portions are the same in terms of their respective appearances.More particularly, the peek view image 314 in contrast to the peekanimation image 310 no longer has the animation features 312 or timedisplay 308, but continues to have an icon 305 identical in appearanceto the icon 304 and icons 307 identical in appearance to the icons 306,except insofar as the icon 305 is now at a location that is movedupwards relative to the location of the icon 304 in the peek animationview (above the location of the uppermost dot of the upper column ofdots of the animation features) and the icons 307 are now at a locationthat is moved vertically downward relative to their previous location inthe peek animation view (below the location of the lowermost dot of thelower column of dots of the animation features).

Further, in the peek view image 314, one or more (in this example,three) text strings lines 316 are also displayed, above the icon 305.The text string lines 316 can include message information or informationcorresponding to one or more past notifications received by theelectronic device 100. The display of these text string lines thusallows the user to “peek” at the notifications that have been received(e.g., recently received) by the electronic device 100, and is thereforethe feature of this manner of operation giving rise to the “peek viewmode” terminology used herein.

Referring additionally to FIGS. 4A, 4B, 4C, and 4D, first, second,third, and fourth additional views 400, 402, 410, and 414, are shown,respectively. The first, second, third, and fourth views 400, 402, 410,and 414 respectively encompass the blank, breathing view, peek animationview, and peek view images 300, 302, 310, and 314 that are shown inFIGS. 3A, 3B, 3C, and 3D, respectively, but also show those images incombination with illustrations of a user—particularly a finger 412 ofthe user's hand—interacting with those images. Because it is envisionedthat the finger 412 of the user has not yet approached the touch screendisplay in the case of the blank image 300 and breathing view image 302,the first additional view 400 of FIG. 4A merely again shows the blankimage 300, and the second additional view 402 of FIG. 4B merely againshows the breathing view image 302. By contrast, with respect to thethird additional view 410 of FIG. 4C, there it is envisioned that theuser has already touched the touch screen display 105 with the user'sfinger 412 and this has given rise to display of the peek animation view310. In particular, it should be noted that the finger 412 is touchingon the icon 304, which constitutes one of the hit areas on the touchscreen display 105 in this example. Further, it is because of (inresponse to) the user's finger 412 touching the hit area that animationfeatures 312 are shown to have appeared.

Finally, at FIG. 4D, the fourth additional view 414 shows the peek viewimage 314 of FIG. 3D and also again shows the user's finger 412 to stillbe proximate to (touching) the touch screen display 105. As representedby arrows 415, while in the peek view mode of operation, the user canfurther provide a gesture (or gesture portion) so as to actuate theelectronic device 100 to perform one of several different actions ortransition to any of several different modes. In the present example,gestural activation of the electronic device 100 in this regardparticularly is not accomplished until, in addition to originallytouching one of the hit areas (again, in this example, one of the icons304 and 306) during the intermediate mode of operation, the user thenfurther: (i) continues to continuously touch that hit area during thetransitioning from the intermediate mode to the peek view mode (e.g.,continues to touch one of the icons 304, 306 as the touch screen displaytransitions from the breathing view 302 to the peek animation view 310and ultimately to the peek view 314); (ii) then manipulates the finger412 to slide upward or downward (e.g., in accordance with either of thearrows 415 of FIG. 4D) until the finger reaches an appropriate one ofthe image portions of the peek view image corresponding to the hit areathat was originally touched (e.g., reaches one of the icons 305, 307 asshown in the peek view 314), and (iii) then releases the hit area byremoving the finger 412 from the touch screen display.

Turning to FIGS. 5, 6A, 6B, 6C, and 6D, a flowchart for a method 500illustrates one embodiment of a method for storing data for a userinteraction during a display session and performing an action by theelectronic device 100 in response to the user interaction. Examples ofuser interaction by the user include viewing or providing a touch inputto the touch screen display 105, pressing an input key 107, providing anaudio or voice input for the microphone 111, or providing a gestureinput (e.g., shaking the electronic device 100). The display session inone example represents a query to the user of the electronic device 100,such as a survey or other prompt for information. In another example,the display session represents a display of a graphic to the user of theelectronic device 100, such as an advertisement or informationalmessage. Other display sessions will be apparent to those skilled in theart. FIGS. 6A, 6B, 6C, and 6D are four example views of a display screenof the electronic device 100 during operation of the method shown inFIG. 5, illustrating a survey as a display session. The user interactionfor the display session occurs, at least in part, during theintermediate or lower-power mode of operation of the electronic device100. Accordingly, the user has fewer distractions (e.g., applications orthe operating system) from the query or graphic. As described above, theprocessor 203 actively controls (502) the touch screen display 105during the higher-power mode of operation.

The processor 203 optionally receives (504) display session informationfor at least one display session during the higher-power mode ofoperation. The display session information comprises at least one of agraphic (e.g., text, images, icons, animated image, video, or acombination thereof), portion of a graphic, action indication, actionicon, keyword, trigger data, or other information for a display session(e.g., a survey or advertisement). In this case, the processor 203stores the display session information in the memory 205 for subsequentaccess by the sensor hub 224 or other components of the electronicdevice 100. In alternative implementations, the display sessioninformation is preconfigured within the electronic device 100 (e.g.,during manufacture or initialization) or stored on a separate memorycard (e.g., a solid state memory card) which is then loaded into theelectronic device 100. The electronic device 100 in one implementationis configured to obtain updates for the display session information. Theelectronic device 100 in one example obtains the updates atpredetermined times (e.g., 2:00 pm, 4:00 pm, etc.), predeterminedintervals (e.g., every hour), or when other criteria are met, such aswhen the electronic device 100 is not in active use by the user orutilization of the electronic device 100 (e.g., utilization of theprocessor 203 or wireless transceivers 201) is below an activitythreshold. In another example, the electronic device 100 obtains theupdates upon the occurrence of an event or signal, such as receiving apush notification from the remote network entity 214. In yet anotherexample, the electronic device 100 obtains updates upon a change in modeof operation for the electronic device 100, for example, upon enteringthe higher-power mode of operation. Other times, schedules, events, orsignals to cause an update will be apparent to those skilled in the art.The electronic device 100 in one example obtains the updated displaysession information through the wireless transceivers 201. In anotherexample, the electronic device 100 obtains the updated display sessioninformation by replacement of a memory card.

During the higher-power mode of operation, the processor 203 isconfigured to discontinue (506) actively controlling the touch screendisplay 105 to enter the lower-power mode of operation (e.g., theintermediate mode). Entry to the lower-power mode is based on a lack ofuser activity, a display timeout, or other criteria that may be used totrigger the electronic device 100 to “sleep,” as will be apparent tothose skilled in the art. Referring to FIG. 6A, the touch screen display105 is shown as a blank image 600 upon entering the lower-power mode ofoperation, analogous to the blank image 300.

During the lower-power mode of operation, the sensor hub 224 isconfigured to provide (508) at least one first control signal to thetouch screen display 105. The sensor hub 224 provides the first controlsignal to cause the touch screen display 105 to activate and display agraphic or action icon for a selected display session, as describedherein. In one example, the sensor hub 224 provides the first controlsignal based on one or more sensor inputs from the sensors 113. Forexample, the sensor hub 224 is configured to provide the first controlsignal when the sensor inputs correspond to the user interacting withthe electronic device 100. The electronic device 100 may have enteredthe intermediate mode when the user placed the electronic device 100 ina pocket, purse, phone case, or other compartment or when the userplaced the electronic device 100 with the front surface 103 facing atable top (not shown) or other surface that obscures or covers the touchscreen display 105. Alternatively, the user may have left the electronicdevice 100 on a table or desk where it has been sitting stationary for aperiod of time. The sensor hub 224 is thus configured to provide thefirst control signal upon an indication that the electronic device 100has been moved after a period of being stationary or upon an indicationthat the touch screen display 105 of the electronic device 100 haschanged to an uncovered state after a period of being in a coveredstate. For example, in response to an input from a proximity sensor,light sensor, accelerometer, or gyroscope of the sensors 113.

In response to the first control signal, the touch screen display 105 isconfigured to activate (510), during the lower-power mode of operation,a first portion of the touch screen display 105 (e.g., using the SmartPartial OLED display technology). The touch screen display 105 thendisplays (512), in the lower-power mode of operation and in response tothe first control signal, a graphic or a first portion of a graphic on afirst area of the touch screen display 105 that is within the firstportion of the touch screen display 105. Referring to FIG. 6B, abreathing view image 602 (analogous to the breathing view image 302) isshown with a graphic 604 on the first area of the touch screen display105. In this case, the display session is a survey that prompts the userof the electronic device 100 for their preference between soda or waterby displaying the graphic 604 that indicates “Soda or Water?” While thegraphic 604 is shown in a lower portion of the touch screen display 105,other portions of the touch screen display 105 may be activated fordisplay. The portion of the touch screen display 105 activated fordisplay may be different for each graphic or display session.

The sensor hub 224 selects a display session based on at least one oftrigger data or keywords associated with the display session, a userprofile associated with the electronic device 100, a location associatedwith the electronic device 100, proximity to a predetermined location(e.g., a restaurant, concert venue, sales kiosk), a time of dayassociated with the electronic device 100, or a selected actionindication from a previous display session. As one example, a survey maybe selected based on a user's location within or near a concert venue.Accordingly, the next time they look at the electronic device 100, theelectronic device 100 provides a survey about which song a band at theconcert venue should play first that night. They can respond to thequestion without waking their device. In other implementations, theelectronic device 100 uses a preconfigured order of display sessions.

A display session in one example includes a preconfigured sequence ofgraphics, for example, a survey (e.g., as shown in FIGS. 6A, 6B, 6C, and6D) which may include a preconfigured sequence of graphics (e.g.,graphics 604, 608, 618) with corresponding hit areas. In anotherexample, the display session information for a display session comprisesa set of graphics from which the sensor hub 224 selects a graphic (e.g.,the graphics 604, 608, 618) for display at various times during thedisplay session. The sensor hub 224 in yet another example selects a setof graphics and action icons for the current display session from a setof available action icons. The sensor hub 224 in one example isconfigured to select graphics, action icons, or other information fordisplay for a current display session from the display sessioninformation previously received (504) and stored in the memory 205. Thesensor hub 224 selects the graphic, action icons, or information basedon at least one of a user profile associated with the electronic device100, a location associated with the current display session (e.g., wherethe electronic device 100 is currently located), proximity to apredetermined location (e.g., a shopping mall or restaurant), a time ofday associated with the current display session, a selected actionindication (described below) from a previous display session, a userinteraction for the current display session, or other information storedby the electronic device 100 (e.g., the user's preferences or searchhistory). In a further example, the sensor hub 224 selects a graphic oraction icon based on trigger data associated with the display session ortrigger data associated with the graphic or action icon. In one example,the sensor hub 224 selects a graphic that has not been previously viewedor interacted with by the user. In the case of a survey, the sensor hub224 may select follow-up questions to previously answered surveys. Forexample, “Soda or Water?” may be followed with “Bottled water or Tapwater?” for a subsequent display session if the user selects “Water” asdescribed below.

The sensor hub 224 is configured to determine (514), in the lower-powermode of operation, whether a first user interaction has occurred thatcorresponds to the graphic 604 during the display of the graphic 604. Asdescribed above, the first user interaction may be a touch input (e.g.,within the hit area), gesture input, audio input, key press, view, orcombination thereof. The sensor hub 224 uses inputs provided by thesensors 113 to determine whether the first user interaction hasoccurred. The sensor hub 224 is configured to store (516), in thelower-power mode of operation, user interaction data for the graphic 604based on the determination (514).

The graphic 604 in one example corresponds to one or more hit areas thatcan be touched by a user during operation in the lower-power mode ofoperation. For a touch input, the user interaction data in one examplecomprises at least one of a duration of the touch input or a location ofthe touch input on the touch screen display 105 (e.g., a hit area). Thelocation may indicate one or more pixels of the touch screen display 105that were touched, a region of the touch screen display 105 that wastouched, or one or more pixels or regions of the graphic 604 that weretouched. For a gesture input, the user interaction data in one exampleindicates a gesture type, such as a shake, rotation (e.g., betweenlandscape and portrait orientations), or other movement. For an audioinput, the user interaction data in one example indicates one or morekeywords detected within the audio input. For a key press input, theuser interaction data in one example indicates which input key 107 waspressed and may further indicate a duration of the key press (or apattern of key presses).

For a view input, the sensor hub 224 determines whether the graphic 604was viewed. The sensor hub 224 activates, in the lower-power mode ofoperation, a sensor 113 (e.g., an imager or front facing camera) of theelectronic device 100. The sensor hub 224 determines, with the imagerand in the lower-power mode of operation, whether a face of a user isdetected. For example, the sensor hub 224 and the imager may determinewhether a pair of eyes or other facial features are detected within aviewing distance of the touch screen display 105. The sensor hub 224stores a view indication that indicates whether the face was detected asthe user interaction data. The sensor hub 224 deactivates the imagerafter the face detection, expiration of a detection timer, or otherdeactivation criteria have been met.

The sensor hub 224 in one example is configured to wait for one or moreuser interactions after (or during) display of the graphic 604. Thesensor hub 224 may further be configured to wait for a combination ofuser interactions, for example, providing a touch input whilesimultaneously viewing the touch screen display 105 or providing a touchinput for a predetermined time (e.g., a first touch threshold). In thiscase, the sensor hub 224 may store user interaction data with anindication that the graphic 604 was not viewed if the duration of thetouch input is below the threshold. Alternatively, the sensor hub 224may omit storing the user interaction data if the threshold is not met.

After storing (516) the user interaction data, the sensor hub 224 mayoptionally end the method 500. For example, where the user interactionrepresents a display of an information message, the user interactiondata may indicate that the information message was viewed (or notviewed) and the sensor hub 224 signals the touch screen display 105 toreturn to the blank image 600. Alternatively, the sensor hub 224 isconfigured to provide (518), in the lower-power mode of operation and inresponse to a first touch input (e.g., of the first user interaction),at least one second control signal to the touch screen display 105. Inone example, the sensor hub 224 provides the second control signal afterthe first touch input has touched the graphic 604 for a duration thatmeets the first touch threshold. In another example, the sensor hub 224provides the second control signal for display of a second graphic whilethe first touch input is substantially maintained such that removal ofthe touch input (e.g., the user removes their finger from the touchscreen display 105) causes the second graphic to be cleared from thetouch screen display 105.

In response to the second control signal and in the lower-power mode ofoperation, the touch screen display 105 activates (520) a second portionof the touch screen display 105 and displays (522) a second graphic on asecond area of the touch screen display 105 that is within the secondportion of the touch screen display 105. Optionally, the touch screendisplay 105 displays (522) a set of action icons, as described belowwith reference to FIG. 7. Referring to FIG. 6C, a peek view image 606(analogous to the peek view image 314) is shown with a second graphic608 on the second area of the touch screen display 105. In this case,the graphic 608 includes a depiction of a soda can and a water glass. Inother display sessions, such as an advertisement, the second graphic isan expanded view of the first graphic. In alternative implementations,the sensor hub 224 may provide an additional control signal to the touchscreen display 105 in order to provide a peek animation image prior todisplay of the peek view image 606, as described above with reference toFIG. 3C.

After or during display of the graphic 608, the sensor hub 224 isconfigured to determine (524), in the lower-power mode of operation,whether a second touch input (e.g., a second user interaction) hasoccurred that corresponds to either the second area (e.g., the secondgraphic 608) or a third area of the touch screen display 105 that isdistinct from the second area. Analogous to the first user interaction,the second user interaction may be a touch input (e.g., within a hitarea), gesture input, audio input, key press, view, or combinationthereof. The third area of the touch screen display 105 in one examplecomprises one or more portions of the touch screen display 105 outsideof the graphic 608. In the present case of a user survey, the user mayinteract with the survey by sliding or swiping their finger up to thegraphic 608 to “select” the depiction of the soda can or the depictionof the water glass (e.g., based on which hit area or pixels correspondto the touch input), or may slide their finger to the third area toindicate that they do not wish to answer the survey.

In response to the second touch input and in the lower-power mode, thesensor hub 224 selects (526) an action indication from a set of actionindications for the display session and stores (528) the actionindication with the user interaction data. The action indication in oneexample provides an indication of the user's selection for the surveyand the sensor hub 224 may optionally end the method 500. For example,where the display session represents a survey, the user interaction datamay indicate that a selection was made (or dismissed) and the sensor hub224 signals the touch screen display 105 to return to the blank image600. While the user interaction data is shown as being stored in steps516 and 528, in alternative implementations only a single step ofstoring is performed, for example, upon completion of the userinteraction or selection of the action indication.

In alternative implementations, the action indication may indicate anaction to be performed (530) by the electronic device 100, as describedherein. In one example, a first action indication corresponds to a touchinput in the second area and a second action indication corresponds to atouch input in the third area. As shown in the example of FIG. 6C, thegraphic 608 corresponds to a first hit area 610 for the soda can (e.g.,within a first partial region of the second portion of the touch screendisplay 105) and a second hit area 612 for the water glass (e.g., withina second partial region of the second portion of the touch screendisplay 105), while a third hit area 614 corresponds to the third area.In this case, the first hit area 610 corresponds to selection of thesoda can, the second hit area 612 corresponds to selection of the waterglass, and the third hit area 614 corresponds to a dismiss actionindication (e.g., the user does not wish to answer the survey).

Referring to FIG. 6D, the action indication in one example correspondsto a display of an image 616 after a user selection for the survey(analogously to the peek view image 606). In the example shown, theimage 616 comprises a graphic 618 that depicts a result for the survey.The result may be dynamically determined by the electronic device 100 ormay be received with the graphic (504).

The graphics 604, 608, and 618 in one example comprise separate graphicelements. In alternative implementations, the graphics 604, 608, and 618may be portions (e.g., primary and secondary portion) of a singlegraphic element to be displayed in a sequence. For example, a singlegraphic element may be sized to occupy a large portion of the touchscreen display 105, but the entire graphic element is displayed orhidden in portions based on the user interaction. In alternativeimplementations, the graphics 604, 608, or 618 may include text, images,icons, animations, or a combination thereof.

After storage of the user interaction data, the electronic device 100 inone example sends the user interaction data to one or more remotenetwork entities 214. For example, the remote network entity 214 may bean application server for an advertiser that places advertisementgraphics on the electronic device 100. The advertiser may use the userinteraction data to determine which advertisements are more effectivethan others at receiving user interactions. The electronic device 100sends the user interaction data analogously to performing the update ofthe graphics (e.g., at predetermined times, predetermined intervals,upon the occurrence of an event or signal, or upon a change in mode ofoperation). Other times, schedules, events, or signals to triggersending of the user interaction data will be apparent to those skilledin the art. In some cases, sending of the user interaction data is aportion of the selected action.

Turning to FIGS. 7A, 7B, and 7C, three example views of a display screenof the electronic device of FIG. 1 during the method 500 are shown,illustrating available actions for a display session. As shown in FIG.7A, a graphic 704 (analogous to graphic 604) comprises an advertisementfor a coupon displayed (512) by the touch screen display 105. Inresponse to a touch input as shown in FIG. 7B, the sensor hub 224provides the second control signal to the touch screen display 105 forthe display (522) of a graphic 706. The graphic 706 comprises a bar codethat may be scanned by a bar code reader (not shown). In this case, thesensor hub 224 may wait for a user interaction (e.g., a touch input oraudio input) before providing a control signal to the touch screendisplay 105 to dismiss or clear the touch screen display 105 beforereturning to the blank image 600. The sensor hub 224 may listen for anaudio beep, such as a beep from a cash register that indicates the barcode has been scanned, before providing the control signal to clear thetouch screen display 105. Thus, the bar code stays “pinned” to the touchscreen display 105 until dismissed by the user.

In another implementation as shown in FIG. 7C, in response to the touchinput, the sensor hub 224 provides the second control signal to thetouch screen display 105 for the display (522) of a set of action icons708, 710, and 712 for the display session. As described above, thesensor hub 224 in one example selects the action icons for display fromthe display session information. In one example, each action icon of theset of action icons corresponds to an action indication for an actionthat may be performed (530) by the electronic device 100. As shown inFIG. 7C, action icon 706 (“Drive”) corresponds to an action forobtaining driving directions, action icon 708 corresponds to an actionfor placing a call, and action icon 710 corresponds to an action for asocial media platform.

As described above, the sensor hub 224 selects (526) the actionindication from the set of action indications based on the userinteraction (e.g., a touch input on a corresponding hit area). Theelectronic device 100 performs a selected action that corresponds to theselected action indication. In a first example, the sensor hub 224performs the selected action in the lower-power mode of operation. In asecond example, the electronic device 100 discontinues the lower-powermode of operation to enter the higher-power mode of operation based onthe selected action. The electronic device 100 (e.g., via the processor203) performs the selected action in the higher-power mode of operation.In a third example, the selected action comprises multiple actions (or acompound action) that are performed partially in the lower-power mode ofoperation and partially in the higher-power mode of operation. In thiscase, actions for the lower-power mode may be performed by the sensorhub 224 during or after the display session, while the actions for thehigher-power mode are performed by the processor 203 during the nexthigher-power mode. In one example, actions for several different displaysessions may be queued for processor 203. The sensor hub 224 maydiscontinue the lower-power mode of operation to enter the higher-powermode of operation based on the selected action, or based upon anothersignal or event that wakes the electronic device 100.

In one example where the display session represents a user survey, afirst graphic comprises a survey prompt for the user survey. In thiscase, at least one action indication of the set of action indicationscorresponds to a survey response indicator for the user survey (e.g.,the user's selection for the survey). If the selected action indicationcorresponds to the survey response indicator, the sensor hub 224 stores,in the lower-power mode of operation, the survey response indicator. Theprocessor 203 in this case sends, in the higher-power mode of operation,a survey response message based on the survey response indicator to aremote server, for example, to provide an advertiser with feedback forthe survey.

The display session in another example represents an offer to the userof the electronic device 100. In this case, at least one actionindication corresponds to a claim to the offer. For example, the sensorhub 224 stores an offer claim indicator to an offer account associatedwith a user of the electronic device 100.

In another example, the display session represents an eventnotification. The event notification allows a user to add a reminder totheir calendar for an event. In this case, at least one actionindication corresponds to a calendar entry indicator for the eventnotification. If the selected action indication corresponds to thecalendar entry indicator, the sensor hub 224 stores the calendar entryindicator in the lower-power mode of operation. The processor 203 sends,in the higher-power mode of operation, a calendar entry message based onthe calendar entry indicator to a remote server for entry into acalendar associated with the electronic device (e.g., the user'scalendar).

The display session in another example corresponds to a subscriptionnotification. In this case, at least one action indication correspondsto a subscription request indicator. If the selected action indicationcorresponds to the subscription request indicator, the sensor hub 224stores, in the lower-power mode of operation, the subscription requestindicator. The processor 203 then sends, in the higher-power mode ofoperation, a subscription request message based on the subscriptionrequest indicator to an email distribution list associated with thesubscription notification. The subscription request message comprises auser email address associated with the electronic device.

In another example, the display session comprises a social media surveythat allows the user to “+1” or “like” a graphic or advertisement. Inthis case, at least one action indication corresponds to a social mediaresponse indicator for the social media survey. If the selected actionindication corresponds to the social media response indicator, thesensor hub 224 stores, in the lower-power mode of operation, the socialmedia response indicator. The processor 203 sends, in the higher-powermode of operation, a social media response message based on the socialmedia response indicator to a remote server. The processor 203 in oneexample prompts the user to select one or more social media platforms touse for the social media response message.

Where the display session comprises an advertisement, in one example anaction indication corresponds to a display of an animated graphic. Asdescribed above with respect to the bar code, the animated graphic maybe pinned to the touch screen display 105 until dismissed by the user.

Another action indication may correspond to a uniform resource locator.In this case, performing the corresponding selected action comprisesconnecting to the uniform resource locator, such as opening a web pagefor an advertiser.

Yet another action indication may correspond to a navigation location.In this case, performing the selected action comprises performing anavigation lookup of the navigation location. For example, theelectronic device 100 opens a navigation application (e.g., GoogleNavigation) and directs the user to the navigation location. Thenavigation location in one example is an absolute location, such as anaddress or GPS coordinates. In another example, the navigation locationis a relative location which instructs the navigation application tofind, for example, a closest location for a restaurant.

Another action indication may correspond to a user application. In thiscase, performing the selected action comprises downloading or launchingthe user application.

Another example of an action indication corresponds to a telephonenumber. In this case, performing the selected action comprises placing acall to the telephone number. In a further example, the actionindication may correspond to several telephone numbers and the sensorhub 224 selects one telephone based on a current time of day. Thisallows the call to be placed to an office during business hours or to avoicemail system after business hours. As described above, the sensorhub 224 may select action indications for a display session. In oneexample, the sensor hub 224 does not select an action indication forplacing a call when the time of the display session is outside of thebusiness hours for receiving the call.

Yet another example of an action indication corresponds to a sharingprompt. For example, the user may wish to share a graphic oradvertisement with a friend. In this case, performing the selectedaction comprises determining a contact address for the sharing prompt,such as an email address or other contact information for the user'sfriend, and then sending a message based on the current display sessionto the contact address.

Another example of an action indication corresponds to a productpurchase. In this case, performing the selected action comprisesinitiating the product purchase. For example, the processor 203 mayconfirm a purchase intent with the user of the electronic device 100. Ina further example, the processor 203 uses billing details stored by theelectronic device 100 (e.g., through Google Wallet) to initiate thepurchase.

While several display sessions and action indications have beendescribed separately, in alternative embodiments a plurality of displaysessions may be combined into a single display session (e.g., withmultiple user interactions). Additionally, multiple actions may becombined and represented by a single action icon or action indication. Adisplay session may use a single action icon or a plurality of actionicons corresponding to action indications.

FIG. 8 is a diagram of one example of user interaction data stored byone or more remote network entities (e.g., the remote network entity214). As described above, a display session may include multiplegraphics, action icons, and associated information (e.g., time of day,location). The electronic device 100 is configured to store one or moreportions of this information with the user interaction data and to sendthe user interaction data to the remote network entity 214. Referring toFIG. 8, a table 800 includes user interaction data from a plurality ofelectronic device 100 representing multiple users.

In the example shown by table 800, each row corresponds to a displaysession on a different electronic device 100 with a column indicatinginformation for display sessions. A Primary Advertisement ID is shown toidentify a first graphic of a display session and a SecondaryAdvertisement ID is shown to identify a second graphic of a displaysession. A set of action indications (“Action 1”, “Action 2”, and“Action 3”) show which action indications were selected and displayedfor a particular display session. A User ID indicates an identity of auser of the electronic device 100 or an ID of the electronic device 100.A Device Model indicates a model name of the electronic device 100. TheTime of Day indicates a time of day at which the display session wasdisplayed by the electronic device 100. The Device Location indicatesGPS coordinates of the electronic device 100 when the display sessionwas displayed. The Wake Reason indicates an event which caused thesensor hub 224 to display the display session, for example, whether aproximity sensor indicates that the electronic device 100 has beenremoved from a pocket or an accelerometer indicates that the electronicdevice 100 has moved after a period of being stationary. A View Durationindicates a length of time that a graphic of the display session wasviewed. An Action Taken indicates which action indicator was selected bythe user, with a separate value (e.g., “0”) used to indicate that noaction was taken. In alternative implementations, additional columns ofuser interaction data may be stored, or columns may be omitted.

FIG. 9 is a chart representing an example histogram 900 for view time ofa graphic. A plurality of view times for graphics were analyzed andcategorized based on a duration of the view time and whether a view timewas more likely to indicate that an action indication was selected bythe user of the electronic device 100. View times 902 were found to bemost likely to result in an action indication selection. Longer viewdurations suggest that a user was more interested in the displaysession, while shorter view durations suggest that a user was lessinterested or, for example, accidentally touched the touch screendisplay 105. In one example, a view duration threshold may beestablished to eliminate unintentional touches and graphic views fromthe data and thus avoid charging advertisers for unintended views. Forexample, this threshold may be set at 1.0 seconds.

The electronic device 100 or remote network entity 214 in one exampleselects different fee rates which may be charged to an advertiser basedon a view duration for a display session of an advertisement. Theelectronic device 100 and user interaction data allow for recording ofwhen a user touches an initial banner advertisement (e.g., the firstgraphic), when the user follows an advertisement (e.g., selects anaction), and thus recording of a success rate for an advertisement. Theelectronic device 100 or remote network entity 214 in one exampleselects a fee for the advertisement based on the success rate for anadvertisement. The electronic device 100 or remote network entity 214 ina further example selects the fee rate based on at least one of thetrigger data for the display session, location, time, or other criteria.In one example for a view duration t, the fee rate is set as follows:

Example 1

If t<1.0, charge $0.00 and If t>1.0, charge $1.00.

As another example:

Example 2

If t<1.0, charge $0.00, if 1.0<t<3.0, charge (t−0.5)/2.5*$1.00, and ift>3.0, charge $1.00.

It can be seen from the foregoing that a method and system for storinguser interaction data from a low-power display session is advantageous.In view of the many possible embodiments to which the principles of thepresent discussion may be applied, it should be recognized that theembodiments described herein with respect to the drawing figures aremeant to be illustrative only and should not be taken as limiting thescope of the claims. Therefore, the techniques as described hereincontemplate all such embodiments as may come within the scope of thefollowing claims and equivalents thereof.

The apparatus described herein may include a processor, a memory forstoring program data to be executed by the processor, a permanentstorage such as a disk drive, a communications port for handlingcommunications with external devices, and user interface devices,including a display, touch panel, keys, buttons, etc. When softwaremodules are involved, these software modules may be stored as programinstructions or computer readable code executable by the processor on anon-transitory computer-readable media such as magnetic storage media(e.g., magnetic tapes, hard disks, floppy disks), optical recordingmedia (e.g., CD-ROMs, Digital Versatile Discs (DVDs), etc.), and solidstate memory (e.g., random-access memory (RAM), read-only memory (ROM),static random-access memory (SRAM), electrically erasable programmableread-only memory (EEPROM), flash memory, thumb drives, etc.). Thecomputer readable recording media may also be distributed over networkcoupled computer systems so that the computer readable code is storedand executed in a distributed fashion. This computer readable recordingmedia may be read by the computer, stored in the memory, and executed bythe processor.

The disclosed embodiments may be described in terms of functional blockcomponents and various processing steps. Such functional blocks may berealized by any number of hardware and/or software components configuredto perform the specified functions. For example, the disclosedembodiments may employ various integrated circuit components, e.g.,memory elements, processing elements, logic elements, look-up tables,and the like, which may carry out a variety of functions under thecontrol of one or more microprocessors or other control devices.Similarly, where the elements of the disclosed embodiments areimplemented using software programming or software elements, thedisclosed embodiments may be implemented with any programming orscripting language such as C, C++, JAVA®, assembler, or the like, withthe various algorithms being implemented with any combination of datastructures, objects, processes, routines or other programming elements.Functional aspects may be implemented in algorithms that execute on oneor more processors. Furthermore, the disclosed embodiments may employany number of conventional techniques for electronics configuration,signal processing and/or control, data processing and the like. Finally,the steps of all methods described herein may be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context.

For the sake of brevity, conventional electronics, control systems,software development and other functional aspects of the systems (andcomponents of the individual operating components of the systems) maynot be described in detail. Furthermore, the connecting lines, orconnectors shown in the various figures presented are intended torepresent exemplary functional relationships and/or physical or logicalcouplings between the various elements. It should be noted that manyalternative or additional functional relationships, physical connectionsor logical connections may be present in a practical device. The words“mechanism”, “element”, “unit”, “structure”, “means”, “device”,“controller”, and “construction” are used broadly and are not limited tomechanical or physical embodiments, but may include software routines inconjunction with processors, etc.

No item or component is essential to the practice of the disclosedembodiments unless the element is specifically described as “essential”or “critical”. It will also be recognized that the terms “comprises,”“comprising,” “includes,” “including,” “has,” and “having,” as usedherein, are specifically intended to be read as open-ended terms of art.The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless the context clearly indicates otherwise.In addition, it should be understood that although the terms “first,”“second,” etc. may be used herein to describe various elements, theseelements should not be limited by these terms, which are only used todistinguish one element from another. Furthermore, recitation of rangesof values herein are merely intended to serve as a shorthand method ofreferring individually to each separate value falling within the range,unless otherwise indicated herein, and each separate value isincorporated into the specification as if it were individually recitedherein.

The use of any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate the disclosedembodiments and does not pose a limitation on the scope of the disclosedembodiments unless otherwise claimed. Numerous modifications andadaptations will be readily apparent to those of ordinary skill in thisart.

1. A method comprising: controlling, by an electronic device and whileoperating in a higher-power mode of operation, a touch screen display ofthe electronic device; prior to the electronic device entering alower-power mode of operation, discontinue controlling, by theelectronic device, the touch screen display; activating, by theelectronic device and while operating in the lower-power mode ofoperation, a portion of the touch screen display; displaying, by theelectronic device and while operating in the lower-power mode ofoperation, a portion of a graphic on a area of the touch screen displaythat is within the first portion of the touch screen display;determining, by the electronic device and while operating in thelower-power mode of operation, whether a user interaction occurs at theportion of the graphic during the display of the portion of the graphic;and storing, by the electronic device and while operating in thelower-power mode of operation, user interaction data for the portion ofthe graphic in response to determining whether the user interactionoccurs at the portion of the graphic.
 2. The method of claim 1, wherein:determining whether the first user interaction occurs at the portion ofthe graphic during the display of the portion of the graphic comprises:responsive to determining that a first touch input occurs at the firstarea, determining, by the electronic device and while operating in thelower-power mode of operation, a duration of the first touch input;storing the user interaction data comprises storing the duration of thefirst touch input for the portion of the graphic.
 3. The method of claim2, wherein storing the user interaction data comprises responsive todetermining that the duration of the first touch input is below a firsttouch threshold, storing an indication that the portion of the graphicwas not viewed.
 4. The method of claim 2, wherein the portion is a firstportion, the method further comprising: responsive to determining thatthe first touch input occurs at the first area, and while operating inthe lower-power mode of operation: activating, by the electronic device,a second portion of the touch screen display; displaying, by theelectronic device, a second portion of the graphic on a second area ofthe touch screen display that is within the second portion of the touchscreen display; and determining, by the electronic device, whether asecond touch input has occurred that corresponds to either the secondarea or a third area of the touch screen display that is distinct fromthe second area, wherein storing the user interaction data furthercomprises storing an action indication that indicates whether the secondtouch input has occurred.
 5. The method of claim 4, wherein: displayingthe second portion of the graphic comprises: displaying the secondportion of the graphic while determining that the first touch inputoccurs at the first area and determining, by the electronic device andwhile operating in the lower-power mode of operation, a duration of thedisplay of the second portion of the graphic; and storing the userinteraction data further comprises storing the duration of the displayof the second portion of the graphic.
 6. The method of claim 5, whereinstoring the user interaction data comprises responsive to determiningthat the duration of the display of the second portion of the graphic isbelow a display threshold, storing an indication that the second portionof the graphic was not viewed.
 7. The method of claim 4, wherein storingthe action indication comprises: storing, by the electronic device, afirst action indication in response to determining that the second touchinput corresponds to the second area; and storing, by the electronicdevice, a second action indication in response to determining that thesecond touch input corresponds to the third area.
 8. The method of claim7, wherein storing the second action indication comprises storing, bythe electronic device, a dismiss action indication.
 9. The method ofclaim 4, wherein: the second portion of the touch screen displaycomprises first and second partial regions; the second area includes afirst hit area in the first partial region and a second hit area in thesecond partial region; and storing the action indication comprisesstoring a first action indication if the second touch input correspondsto the first hit area and storing a second action indication if thesecond touch input corresponds to the second hit area.
 10. The method ofclaim 4, wherein the first portion of the graphic is a primary portionof the graphic and the second portion of the graphic is a secondaryportion of the graphic.
 11. The method of claim 1, wherein activatingthe portion of the touch screen display comprises activating the portionof the touch screen display in response to at least one of: determiningthat the electronic device moved after a period of being stationary; ordetermining that the touch screen display of the electronic devicechanged to an uncovered state after a period of being in a coveredstate.
 12. The method of claim 1, further comprising sending, by theelectronic device, the user interaction data to a remote network entity.13. The method of claim 1, wherein determining whether the first userinteraction has occurred comprises: activating, by the electronic deviceand while operating in the lower-power mode of operation, an imager ofthe electronic device; and determining, by the electronic device withthe imager and while operating in the lower-power mode of operation,whether a face of a user is detected, wherein storing the userinteraction data comprises storing a view indication that indicateswhether the face was detected.
 14. An electronic device comprising: areceiver configured to receive wireless communication signals; at leastone memory device; a touch screen display configured to, while theelectronic device operates in a lower-power mode of operation: activate,in response to at least one first control signal, a first portion of thetouch screen display; display, in response to the at least one firstcontrol signal, a first portion of a graphic on a first area of thetouch screen display that is within the first portion of the touchscreen display; activate, in response to at least one second controlsignal, a second portion of the touch screen display; and display, inresponse to the at least one second control signal, a second portion ofthe graphic on a second area of the touch screen display that is withinthe second portion of the touch screen display; at least one applicationprocessor configured to, while the electronic device operates in ahigher-power mode of operation: control the touch screen display;receive and store the graphic in the at least one memory device; andprior to the electronic device operating in the lower-power mode ofoperation, discontinue controlling the touch screen display; at leastone processing device configured to, while the electronic deviceoperates in the lower-power mode of operation: control the touch screendisplay; provide the at least one first control signal to the touchscreen display; determine whether a first touch input that correspondsto the first area occurs during the display of the first portion of thegraphic; provide, in response to the first touch input, the at least onesecond control signal to the touch screen display; and store, userinteraction data for the graphic, wherein the user interaction datacomprises at least one of a duration of the first touch input, aduration of the display of the second portion of the graphic, or anaction indication that corresponds to the first touch input.
 15. Theelectronic device of claim 14, wherein the at least one processingdevice is further configured to, while the electronic device operates inthe lower-power mode of operation: select a first action indication inresponse to determining that the touch input changed from occurringwithin the first area to occurring within the second area; select asecond action indication in response to determining that the touch inputchanged from occurring within the first area to occurring within a thirdarea of the touch screen display, wherein the third area is within thesecond partial region and distinct from the second area; and store theselected action indication.
 16. The electronic device of claim 15,wherein: the at least one application processor is further configuredto, while the electronic device operates in the higher-power mode ofoperation, receive and store a plurality of graphics in the at least onememory device; and the at least one processing device is furtherconfigured to, while the electronic device operates in the lower-powermode of operation: select the graphic from the plurality of graphicsbased on graphic selection criteria associated with the electronicdevice; and store the graphic selection criteria with the userinteraction data for the graphic.
 17. The electronic device of claim 16,wherein the at least one processing device is further configured to,while the electronic device operates in the lower-power mode ofoperation and in response to detecting a trigger action: select thegraphic from the plurality of graphics; provide the at least one firstcontrol signal to the touch screen display, wherein the graphicselection criteria comprises at least one of: a trigger action type thatindicates the trigger action, a time of the trigger action, or a date ofthe trigger action.
 18. An electronic device comprising: a touch screendisplay; an application processor configured to control the touch screendisplay while the electronic device operates in a higher-power mode ofoperation; a sensor hub configured to control the touch screen displaywhile the electronic device operates in an lower-power mode ofoperation; and a non-transitory memory comprising instructions thatconfigure the electronic device to: control, with the applicationprocessor while operating in the higher-power mode of operation, thetouch screen display; prior to entering a lower-power mode of operation,discontinue controlling, with the application processor, the touchscreen display; provide, with the sensor hub and while operating in thelower-power mode of operation, at least one first control signal to thetouch screen display for activating a portion of the touch screendisplay and displaying a portion of a graphic on a first area of thetouch screen display that is within the portion of the touch screendisplay; determine, with the sensor hub and while operating in thelower-power mode of operation, whether a first user interaction thatcorresponds to the portion of the graphic has occurred during thedisplay of the portion of the graphic; and store, with the sensor huband while operating in the lower-power mode of operation, userinteraction data for the portion of the graphic based on the first userinteraction determination.
 19. The electronic device of claim 18,wherein the portion is a first portion, and the non-transitory memorycomprises further instructions that configure the electronic device to:determine, with the sensor hub, that a first touch input has occurred atthe first area; determine, with the sensor hub, a duration of the firsttouch input; store, with the sensor hub, the duration of the first touchinput with the user interaction data; responsive to the first touchinput, provide, with the sensor hub and while operating in thelower-power mode of operation, at least one second control signal to thetouch screen display for activating a second portion of the touch screendisplay and displaying a second portion of the graphic on a second areaof the touch screen display that is within the second portion of thetouch screen display; determine, with the sensor hub and while operatingin the lower-power mode of operation, whether a second touch input hasoccurred that corresponds to the second area or a third area of thetouch screen display that is distinct from the second area; and store,with the sensor hub and while operating in the lower-power mode ofoperation, the user interaction data with an action indication thatindicates whether the second touch input has occurred and in which areait occurred.
 20. The electronic device of claim 18, wherein theapplication processor is configured to send, via a network and to aremote network entity, the user interaction data.